{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lab 16 - Bootstrap and confidence intervals\n", "\n", "For this lab, we will use data from a sample of babies born between 1960 and 1967 in California hospitals. The data can be downloaded from [here](https://www.stat.berkeley.edu/~statlabs/data/babies.data). For reference, this file and a description of the columns is on the page [https://www.stat.berkeley.edu/~statlabs/labs.html](https://www.stat.berkeley.edu/~statlabs/labs.html), under the heading \"Maternal Smoking and Infant Health II\". \n", "\n", "This lab closely follows [Chapter 13.3 in Computational and Inferential Thinking: The Foundations of Data Science](https://www.inferentialthinking.com/chapters/13/3/Confidence_Intervals.html) by Ani Adhikari and John DeNero.\n", "\n", "Open the data file in TextEdit (found under Applications in Finder). What do you notice about it? How are the columns separated? \n", "\n", "The columns are: \n", "\n", "- bwt: Birth weight in ounces (999 unknown)\n", "- gestation: Length of pregnancy in days (999 unknown)\n", "- parity: 0= first born, 9=unknown\n", "- age: mother's age in years\n", "- height: mother's height in inches (99 unknown)\n", "- weight: Mother's prepregnancy weight in pounds (999 unknown)\n", "- smoke: Smoking status of mother: 0=not now, 1=yes now, 9=unknown\n", "\n", "First, let's import the necessary libraries." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the data file, the columns are separated by whitespace (ex. spaces, tabs, etc.) instead of by commas, as usual. We can still use `pd.read_csv()` but need to add the parameter `sep = \"\\s+\"` to tell the function that the columns are separated one or more whitespaces. \n", "\n", "Read `babies.data` into the dataframe `babies` using this parameter `sep = \"\\s+\"` and check it worked correctly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How big is the sample? We will use this later." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Pattern:\n", "\n", "df.shape[0]\n", "
\n", "\n", "## 95% confidence interval for median birth weight to number of gestational days ratio\n", "\n", "We would like to understand the relation between birth weight, which is given in ounces, and the number of gestational days. Smaller babies often need more medical care, and it would be helpful to be able to estimate a baby's weight before it is born.\n", "\n", "We'll look at the ratio of birth weight to number of gestational days. For the first row, this ratio is:\n", "$\\frac{\\text{120 oz}}{\\text{284 days}} \\approx \\text{0.4225 ounces per day}$\n", "\n", "Let's make a new column of the birth weight to number of gestational days ratio. Recall from Lab 2 that we can do column-wise math operations." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Answer\n", "\n", "babies[\"Ratio BW/GD\"] = babies[\"bwt\"]/babies[\"gestation\"]\n", "\n", "
\n", "\n", "Plot a histogram of these ratios:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see in the histogram that there are some very low and very high ratios, which are *outliers*. Because of these extreme values we should use a robust measure of center, namely the median. Compute the median of the ratios." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Pattern:\n", "\n", "df[\"column_name\"].median()\n", "\n", "
\n", "\n", "This is the median ratio of the sample, but what is the median ratio of the population? We will estimate a range for it, called the *95% confidence interval*, by creating more samples from the sample we have. This process of creating more data with the same distribution as our existing data is called *bootstrapping*.\n", "\n", "Above we found that our sample size is 1236. Sampling *with replacement* means that we can choose the same row more than once (the row is replaced once we have chosen it). \n", "\n", "Steps for computing the 95% confidence interval:\n", "1. Take a random sample of size 1236 from our sample, by sampling *with replacement*, and compute the median of the new sample.\n", "2. Repeat Step 1 until we have computed 5000 sample medians, which we save in a list.\n", "3. Compute the middle 95% of these medians.\n", "\n", "#### Step 1: Take a random sample of size 1236 from our sample, by sampling *with replacement*, and compute the median of the new sample.\n", "\n", "To re-sample with replacement from our original sample, we still use the `.sample()` function, but add the parameter `replace = True`.\n", "\n", "Take a sample of size 1236 with replacement from the `babies` dataframe, and store it in the dataframe `bootstrap_sample`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Answer\n", "\n", "bootstrap_sample = babies.sample(1236, replace = True)\n", "
\n", "\n", "Check that `bootstrap_sample` was created correctly by displaying it." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Take the median ratio of the sample:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How does this median compare to the median of the original data? The two medians should be similar.\n", "\n", "#### Step 2: Take 5,000 samples and compute the median for each as in Step 1. Store these medians in a list.\n", "\n", "This step is similiar to what we have done in previous labs, just with a different computation inside the loop.\n", "\n", "The pseudo-code is:\n", "\n", " new_list = []\n", " for i in range(num_of_loops):\n", " take bootstrap sample\n", " compute median of ratio in bootstrap sample\n", " add median to new_list\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plot the medians as a histogram, remembering to convert the list into a Pandas Series first:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Pattern:\n", "\n", "pd.Series(median_list).hist()\n", "
\n", "\n", "What do you notice about the distribution? What is the range of the distribution? \n", "\n", "#### Step 3: Compute the middle 95% of these medians.\n", "\n", "To get the interval containing the middle 95% of our sample medians, we compute the 2.5 percentile, which is the sample median such that 2.5% of the sample medians are below it, and the 97.5 percentile, which is the sample median such that 97.5% of the sample medians are below it.\n", "\n", "Pandas has a function for computing *quantiles* which are the percentiles divided by 100. ie. the 2.5 percentile is the 0.025 quantile and the 97.5 percentile is the 0.975 quantile.\n", "\n", "Type `pd.Series(medians).quantile(0.025)` below to compute the 0.025 quantile." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now compute the 0.975 quantile:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The 95% confidence interval is [0.425,0.431] from my calculations. Yours may differ slightly because we use random sampling to compute this interval.\n", "\n", "Technically this interval is an *approximation* of the 95% confidence interval that you may have seen in another statistics class.\n", "\n", "*Interpretation of the 95% confidence interval*: If we repeated this procedure for computing the 95% confidence interval on many different original samples from the population, then 95% of the intervals would contain the true population median.\n", "\n", "This method for computing the 95% confidence interval also works for means and proportions. It may not work well if:\n", "- the sample size is small (say less than 10 or 15)\n", "- the population distribution of the statistic is not roughly bell shaped\n", "- we are trying to estimate a statistic that is greatly influenced by outliers or rare elements, like the maximum data value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 95% confidence interval of the proportion of mothers who smoke\n", "\n", "Now we will use bootstrapping to compute the 95% confidence interval of the proportion of mothers who smoke.\n", "\n", "First estimate the proportion of mothers who smoke in our sample, by computing:\n", "$$\\text{proportion of mothers who smoke} = \\frac{\\text{# of birth where mother smoked}}{\\text{total # of births}}$$\n", "\n", "Calculate the number of births where mother smoked:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Hint:\n", "Use a filter to find the rows with 1 in the `smoke` column. \n", "
\n", "\n", "We computed the total number of births at the start of the lab.\n", "\n", "Compute the proportion of mothers who smoke:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now use bootstrapping to compute a 95% confidence interval for the proportion of mothers who smoke.\n", "\n", "#### Step 1 and 2: Re-sample from the sample and compute the proportion of mothers who smoke for each bootstrap sample.\n", "\n", "The pseudo-code is:\n", "\n", " new_list = []\n", " for i in range(num_of_loops):\n", " take bootstrap sample\n", " compute proportion of mothers who smoke in bootstrap sample\n", " add proportion to new_list\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Answer:\n", "\n", "proportions = []\n", "for i in range(5000):\n", " bootstrap_sample = babies.sample(1236, replace=True)\n", " smoke_filter = bootstrap_sample[\"smoke\"] == 1\n", " num_smoke = smoke_filter.sum()\n", " proportion = num_smoke/1236\n", " proportions.append(proportion)\n", " \n", "
\n", "\n", "Plot a histogram of the proportions, remembering to make the list into a Pandas Series first." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 3: Compute the 95% confidence interval for the proportions\n", "\n", "Find the 0.025 quantile and the 0.975 quantile to get the interval." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Pattern:\n", "pd.Series(list_of_sample_statistic).quantile(desired_quantile)\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What is your 95% confidence interval?\n", "\n", "*Interpretation of the 95% confidence interval*: If we repeated this (random) procedure for computing the 95% confidence interval multiple times, then approximately 95% of the confidence intervals computed would contain the true proportion of mothers who smoke in the population." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Challenges: \n", "- Find the 95% confidence interval for the age of the mothers in the sample.\n", "- Find the 95% confidence interval for the mean birth weight in the sample." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 2 }